Open
Conversation
* fix: revert skip validation * fixed logic * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * made updated change * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * changed to === * fixed implementation * added comment * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * refactored boolean * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fixed bugs * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * linted files * removed console.log * fixed unit tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * added test Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
…2074) * test: Add system-tests for resumable multi-chunk uploads * test: streamline multi-chunk upload system tests * chore: verify `Content-Range`: `bytes ${this.offset}-*/*` behavior * style: typo * style: typo * fix: determine content-length before attempting multi-chunk upload * test: Use bigger file for multi-chunk test since chunks need to be large * feat: dynamically determine Content-Length when unspecified * feat: dynamically determine Content-Length when unspecified * docs: clarifications, make it easier for the future team * docs: Another clarification
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* refactor(deps): remove arrify * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* feat(crc32c): Convenient Method For Reading Files * fix: `fs/promises` for Node 12 * fix: Use `unlink`, `rm` isn't available on Node 12 * fix: Use `unlink`, `rm` isn't available on Node 12
#2094) * fix: remove async from final function which causes double invocation of callback * add catch clause
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* feat: support autoclass * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * added documentation * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fixed documentation * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * removed log statements Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
…will be rotated (#2112) * build: have Kokoro grab service account credentials from secret that will be rotated Source-Link: https://togithub.com/googleapis/synthtool/commit/4a0230eb8dc497f36fd3839e6144982131f30a9d Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:f59941869d508c6825deeffce180579545fd528f359f549a80a18ec0458d7094
* feat: implement parallel operations (#2067) * feat: implement parallel operations * add more parallel operations * add header to test file * update import of fs/promises * fix pathing on windows, fix mocking of fs promises * add jsdoc headers to class and uploadMulti * add jsdoc comments to remaining functions * update comment wording * add experimental jsdoc tags * feat: add directory generator to performance test framework * clarify variable names and comments * capitalization * wip: transfer manager performance tests * feat: merged in application performance tests (#2100) * fix: fixed many bugs (#2102) * fix: cleaning up bugs * fix: fixed many bugs * fix: more work on transfer manager perf metrics (#2103) * fix: more work on transfer manager perf metrics * fix unit tests for tm * fix: performance test refactoring, comments (#2104) * refactor: refactor constants (#2105) * refactor: refactor constants * bug fixes * bug fixes * bug fixes * linter fixes, download to disk for performance test * rename transfer manager functions * remove callbacks from transfer manager * add more experimental tags, update comments * change signature of downloadManyFiles to accept array of strings or a folder name * linter fix * add transfer manager samples and samples tests Co-authored-by: Sameena Shaffeeullah <shaffeeullah@google.com> * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add crc32c validation option to chunked download, cleanup naming (#2110) * feat: add crc32c validation option to chunked download, cleanup naming in perf tests * close file in finally block Co-authored-by: Sameena Shaffeeullah <shaffeeullah@google.com> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
…will be rotated for system tests (#2121) Source-Link: https://togithub.com/googleapis/synthtool/commit/abbc97db69a57dcb991ba97ef503305b701ffb3a Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:fe04ae044dadf5ad88d979dbcc85e0e99372fb5d6316790341e6aca5e4e3fbc8
…2124) This is a follow-up to PR googleapis/nodejs-storage#2123 in hope to address both README.md elaborate documentation updates as well as in-code documentation
#2118) * feat: add ability to upload entire directory, add samples for upload directory / download folder * linter fix * update phrasing of comments * update comment in downloadFolderWithTransferManager sample * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
…ationmatch not ifgenerationmatch (#2131)
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* test: add output format for SSB to performance tests * rename SSB to cloud monitoring * reword comments for additional clarity * adjust threshold for chunked downloads in order to support workload 2 * fix a few typos, consume async generator correctly
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* feat: Support Node 18+ * chore: Additional Node 18 upgrades * chore: don't change test fixtures * feat: Move kokoro directories to 18 * chore: Update checkout and setup-node to v4 - https://github.com/actions/checkout/releases/tag/v4.0.0 - https://github.com/actions/setup-node/releases/tag/v4.0.0 Source-Link: googleapis/synthtool@c19dd80 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:8060aba1f6d5617d08091767141ab2a99ea1ccbd9371fd42ffc208c5329caa73 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Denis DelGrosso <85250797+ddelgrosso1@users.noreply.github.com>
…2668) * fix: 🐛 fix the issue 2667, do not mutate object given to options argument in file download method * test: 🧪 add the test for issue 2667, and fix another faulty test * style: 🎨 fix the lint error in test
* chore: fix ignore regex in owlbot.yaml * re-add samples to owlbot.py * fix regex format
* Chore: Update move logic to support non-HNS buckets The existing logic for file move operations was implicitly tied to features available only in Hierarchical Namespace (HNS) enabled buckets. This update refactors the underlying move implementation to correctly handle the different API requirements and constraints of **non-HNS buckets**. * refactor: system test * add more assert * bug fix * addressing comments
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* samples: Add Soft Delete policy and object management samples Adds complete code samples demonstrating the following operations: - storage_disable_soft_delete - storage_get_soft_delete_policy - storage_set_soft_delete_policy - storage_list_soft_deleted_objects - storage_list_soft_deleted_object_versions - storage_restore_object * remove conflict * bug fix * refactor: Use literal for retention duration and improve test assertion - Removed the SOFT_DELETE_RETENTION_SECONDS constant, replacing it with the literal value (604800) and adding a comment. - Updated the listSoftDeletedObjectVersions test to assert on the object generation number, improving test coverage and reflecting the logging change. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
chore(python): remove configure_previous_major_version_branches from python post processor Source-Link: googleapis/synthtool@6702a34 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:bb3dc6c75c5f3e1a8928474f2dc8d9bbfa7b809045515764b9ff48c070cf2321 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* feat(listBuckets): Add support for returning partial success Implements the optional parameter `returnPartialSuccess`. If set to `true` and the API response contains an `unreachable` array, the function will return a successful callback (`err: null`) and pass the `unreachable` array as the 5th argument. This allows users to process the available buckets without the entire operation failing on soft errors. * docs(samples): Add listBucketsPartialSuccess sample New sample for partial success in ListBuckets (storage_list_buckets_partial_success) * code refactor * minor fix * Ensure positional consistency for optional callback results * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Enable partial success via placeholder objects * bug fix * addressing commends * refactor: Simplify partial success placeholder logic * addressing comments * Revert "addressing comments" This reverts commit 10a7e87a37bb207164b15db9b9a0139c2b35a6df. * addressing comments * chore: Clean up unreachable placeholder creation loop * refactor: test case --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@7751597 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:a1e45c1b53fceb5fa7fd5c49cf898048c210eb07001a3355548e7a4545200ee8 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* feat: Enable full object checksum validation on JSON path Adds validation for client-provided (pre-calculated) and client-calculated CRC32C and MD5 hashes when the final upload request is made via the JSON API path (status 200). This ensures consistency checks are performed even when the `Upload` stream is finalized, preventing silent data corruption if the server-reported hash (in the response body) mismatches the client's expected hash. * fix: duplicate codes removed * added unit test cases * fix: Resolve checksum validation failures and add full test suite Adds the 'Validation of Client Checksums Against Server Response' test suite. Fixes test failures in client-provided hash scenarios by updating mock responses to ensure server-reported checksums match the client's expected values. * code refactor * refactor: Combine and parameterize checksum validation tests Refactors four duplicate test cases (CRC32C/MD5 success and failure) into a single, parameterized test block within the 'Validation of Client Checksums Against Server Response' suite. This improves test clarity and reduces code duplication by dynamically generating test scenarios for post-upload hash validation. * addressing comments * bug fix * added system test * fix system test * bug fix * fix: Address stream consumption, checksum, and multi-part session hangs This commit introduces several stability fixes for the ResumableUpload class: 1. **Fixes Timeouts in Unit Tests:** Updates `makeRequestStream` mocks to fully drain the request body stream, resolving stream consumption deadlocks and timeouts in `#startUploading` unit tests. 2. **Fixes Multi-Part Hang:** Correctly finalizes the `pipeline` for partial chunks (`isPartialUpload=true`) by calling `pipelineCallback()` immediately after successful chunk upload, preventing indefinite hangs in multi-session tests. 3. **Fixes Single-Chunk Checksum Missing Header:** Applies the `X-Goog-Hash` header unconditionally in single-chunk mode if a validator is present, ensuring checksum validation is active even when `contentLength` is unknown. * code refactor * code refactor * refactor: streamline checksum and resumable upload logic Simplify `HashStreamValidator._flush` by utilizing `md5Digest` getter.
Source-Link: googleapis/synthtool@36ebc33 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:1c50961ec998cd24968d5b2628e8d88a36b612ea881e71bfd171c93cc9d08b9e Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Skips tests that attempt to assign public or authenticated user roles to IAM policies. These tests are currently failing with 403 errors due to environment permission restrictions.
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@1de1899 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-nodejs:latest@sha256:1bf17970ae27b8ff1cad706a4d68a3592723575d3a5a9de6afe3d2726eed7f2d Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* fix: append bucket name to V4 policy URL in emulator mode * fix: Set temporary env variables for testing and cleanup
* chore: replace old teams with cloud-sdk-nodejs-team and gcs-team b/478003109 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| .filter(x => x!.trim()) // Limit to non-empty strings. | ||
| .map(uriComponent => { | ||
| const trimSlashesRegex = /^\/*|\/*$/g; | ||
| return uriComponent!.replace(trimSlashesRegex, ''); |
Check failure
Code scanning / CodeQL
Polynomial regular expression used on uncontrolled data High
| reqOpts.uri = uriComponents | ||
| .map(uriComponent => { | ||
| const trimSlashesRegex = /^\/*|\/*$/g; | ||
| return uriComponent.replace(trimSlashesRegex, ''); |
Check failure
Code scanning / CodeQL
Polynomial regular expression used on uncontrolled data High
| if (!PROTOCOL_REGEX.test(url)) { | ||
| url = `https://${url}`; | ||
| } | ||
| return url.replace(/\/+$/, ''); // Remove trailing slashes |
Check failure
Code scanning / CodeQL
Polynomial regular expression used on uncontrolled data High
Comment on lines
+13
to
+21
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 14 | ||
| - run: node --version | ||
| - run: cd handwritten/storage && npm install | ||
| - run: cd handwritten/storage && npm run conformance-test |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.